Systems and methods for providing media recommendations

ABSTRACT

Systems and methods are described for performing an action related to an identifier for a recommended media asset presented to a user, based on a detected emotional indicator of the user. The identifier for the initial recommended media asset is generated for presentation to the user, and one or more images of the user are captured while generating for presentation the identifier for the initial recommended media asset to the user. An emotional indicator of the user is detected based on the one or more captured images, and an action related to the identifier for the initial recommended media asset is performed based on the detected emotional indicator.

BACKGROUND

This disclosure relates to recommending media assets to a user, and moreparticularly, to systems and methods for identifying users by facialrecognition and presenting media content recommendations to the users,and systems and methods for performing an action related to anidentifier of recommended media content presented to a user, based on adetected emotional indicator of the user.

SUMMARY

Modern media distribution systems enable a user to access more mediacontent than ever before. However, given the large variety of mediaproviders and media assets available to a user, it may a challengingtask for users of media services (e.g., cable, broadcast, satellite,over-the-top provider) to efficiently locate content he or she isinterested in.

In one approach, recommended content may be provided to a user based onother content the user has consumed. However, many viewers preferconsuming content with friends and family, and recommended content basedon viewing habits of only one of the users may not be useful in findingcontent that would be enjoyable to multiple users with differentinterests. In another approach, a user may be permitted to scrollthrough various recommended content items in order to locate a contentitem he or she is interested in. However, such approach merely enables auser to passively navigate a static set of recommended content items,without taking into account whether the user, in real time, isinterested in any of the content items. This may frustrate the user,such that the user may decide not to consume media at all. In suchinstance, the next time the user attempts to consume content he or shemay merely be provided with the same recommendations that did notinterest him or her (e.g., since his or her viewing history isunchanged).

In some embodiments, to overcome one or more of these problems, systemsand methods are provided herein for presenting a graphical userinterface (GUI) including identifiers for media assets recommended foreach of multiple users detected to be in the vicinity of the userequipment. A content recommendation application identifies, using facialrecognition, a plurality of users (including a first user and a seconduser) in a vicinity of user equipment, and determines a firstrecommended media asset for the first user and a second recommendedmedia asset based on respective user profiles of the first and seconduser. The content recommendation application generates for presentationthe GUI including a first identifier selectable to access the firstrecommended media asset and a second identifier selectable to access thesecond recommended media asset, and in response to receiving selectionof the first identifier or the second identifier, generates forpresentation the recommended media asset associated with the selectedidentifier. Such aspects allow simultaneous presentation of recommendedmedia assets for each user that is interested in consuming media, tofacilitate selection of content each user can enjoy. In addition, evenif one of the users (e.g., the second user) has never used a device(e.g., a television at the first user's home) on which content is to beconsumed, recommended content for such user can conveniently bepresented without requiring any effort on the part of the user.

In some embodiments, to overcome one or more of the above problems,systems and methods are also provided herein for performing an actionrelated to an identifier for a recommended media asset based on adetected emotional indicator of a user. A content recommendationapplication may generate for presentation to a user an identifier for aninitial recommended media asset, and capture one or more images of theuser while generating for presentation the identifier for the initialrecommended media asset to the user. The content recommendationapplication may detect an emotional indicator of the user based on theone or more captured images, and perform, based on the detectedemotional indicator, an action related to the identifier for the initialrecommended media asset. Such aspects enable a suitable action (e.g.,presenting an identifier for an updated recommended media asset,selecting the identifier for a media asset, presenting a preview of therecommended media asset, refraining from updating the media asset, etc.)to be dynamically performed based on an emotion being exhibited by theuser (e.g., while reviewing one or more identifiers for recommendedmedia assets).

In some embodiments, the content recommendation application maydetermine a third recommended media asset for the first user and thesecond user based on the user profile of the first user and the userprofile of the second user. A third identifier selectable to access thethird recommended media asset may be generated for presentation, and thethird recommended media asset may be generated for presentation inresponse to receiving selection of the third identifier.

In some aspects of this disclosure, the GUI may further include a firstcategory identifier associated with a first plurality of recommendedmedia assets determined based on the user profile of the first user,where the first plurality of recommended media assets includes the firstrecommended media asset. The GUI may further include a second categoryidentifier associated with a second plurality of recommended mediaassets determined based on the user profile of the second user, wherethe second plurality of recommended media assets may include the secondrecommended media asset. In some embodiments, the GUI may furtherinclude a third category identifier associated with a third plurality ofrecommended media assets (including the third recommended media asset)determined based on the user profile of the first user and the userprofile of the second user.

In some embodiments, at least one recommended media asset included inthe third plurality of recommended media assets may not be included inthe first plurality of recommended media assets and the second pluralityof recommended media assets. User profiles of each user may include aviewing history of the user, and the user profiles may be updated basedon selection of the third identifier.

The GUI may further include a first view associated with the first userin which the first identifier is presented more prominently than thesecond identifier and the third identifier, a second view associatedwith the second user in which the second identifier is presented moreprominently than the first identifier and the third identifier, and athird view in which the third identifier is presented more prominentlythan the first identifier and the second identifier. The contentrecommendation application may generate for presentation a selectableoption to navigate from the first view to the second view (and/or fromthe first view to the third view, and/or the second view to the thirdview, and/or vice versa).

In some embodiments, the content recommendation application may detectwhether the second user remains within the vicinity of the userequipment, and the content recommendation application may, in responseto determining that the second user has not been within the vicinity ofthe user equipment for a predefined period of time, cease the generatingfor presentation of the identifier of the second recommended media asset(and/or the third recommended media asset).

In some aspects of this disclosure, the content recommendationapplication may determine that the detected emotional indicatorindicates the user is not interested in the initial recommended mediaasset, and the action to be performed based on the detected indicatormay comprise generating for presentation an identifier for an updatedrecommended media asset. The updated recommended media asset associatedwith the identifier may be determined based on a retrieved user profileand the detected emotional indicator.

In some embodiments, the content recommendation application maydetermine the detected emotional indicator indicates the user isinterested in the initial recommended media asset, and the action maycomprise selecting the identifier for the initial recommended mediaasset.

Detecting the emotional indicator of the user may comprise identifyingat least one of a facial expression of the user or body language of theuser. The content recommendation application may detect an initialemotional indicator of the user prior to generating for presentation theidentifier for the initial recommended media asset, where the identifierfor the initial recommended media asset is generated for presentationbased on the initial emotional indicator of the user.

In some embodiments, the content recommendation application may generatefor presentation a plurality of identifiers for respective initialrecommended media assets, where the plurality of identifiers for therespective initial recommended media assets includes the identifier forthe initial recommended media asset. The emotional indicator of the usermay be detected while receiving a command from the user to scrollthrough the identifiers of the plurality of initial recommended mediaassets.

In some embodiments, the content recommendation application may store inmemory a table of facial characteristics and corresponding emotionalindicators, and detecting the emotional indicator based on the one ormore captured images may comprise identifying facial characteristics ofa face of the user in the one or more captured images; comparing theidentified facial characteristics to the stored facial characteristics;determining, based on the comparison, whether the identified facialcharacteristics match the stored facial characteristics; and in responseto determining the identified facial characteristics match the storedfacial characteristics, determining the emotional indicator of the useris the emotional indicator that corresponds to the matched facialcharacteristic.

The content recommendation application may identify a plurality of usersin a vicinity of user equipment, where the user is included in theplurality of users, and at least one of the plurality of users isdetected by facial recognition; capture one or more images of theplurality of users while generating for presentation the identifier forthe initial recommended media asset to the users; detect, based on thecaptured images, respective emotional indicators of the plurality ofusers while generating for presentation the identifier for the initialrecommended media asset to the users; and determine an aggregateemotional indicator of the plurality of users; wherein the action to beperformed is determined based on the aggregate emotional indicator ofthe plurality of users. In some embodiments, detecting respectiveemotional indicators of the plurality of users comprises identifying atleast one of the facial expressions of the users or body language of theusers.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the present disclosurewill be apparent upon consideration of the following detaileddescription, taken in conjunction with the accompanying drawings, inwhich like reference characters refer to like parts throughout, and inwhich:

FIG. 1 shows an example of generating identifiers for recommended mediaassets for multiple users detected to be in a vicinity of userequipment, in accordance with some embodiments of this disclosure;

FIG. 2A shows an example of performing, based on a detected emotionalindicator, an action related to an identifier for a recommended mediaasset, in accordance with some embodiments of this disclosure;

FIG. 2B shows an example of performing, based on a detected emotionalindicator, an action related to an identifier for a recommended mediaasset, in accordance with some embodiments of this disclosure;

FIG. 2C shows an example of performing, based on a detected emotionalindicator, an action related to an identifier for a recommended mediaasset, in accordance with some embodiments of this disclosure;

FIG. 2D shows an example of performing, based on a detected emotionalindicator, an action related to an identifier for a recommended mediaasset, in accordance with some embodiments of this disclosure;

FIG. 3 is a block diagram of an illustrative system in accordance withsome embodiments of the disclosure;

FIG. 4 is another block diagram of an illustrative system in accordancewith some embodiments of the disclosure;

FIG. 5 is a flowchart of a detailed illustrative process for generatingidentifiers for recommended media assets for multiple users detected tobe in a vicinity of user equipment, in accordance with some embodimentsof this disclosure;

FIG. 6 is a flowchart of a detailed illustrative process for generatingidentifiers for recommended media assets for multiple users detected tobe in a vicinity of user equipment, in accordance with some embodimentsof this disclosure;

FIG. 7 is a flowchart of a detailed illustrative process for performing,based on a detected emotional indicator, an action related to anidentifier for a recommended media asset, in accordance with someembodiments of this disclosure; and

FIG. 8 is a flowchart of a detailed illustrative process for performing,based on a detected emotional indicator, an action related to anidentifier for a recommended media asset, in accordance with someembodiments of this disclosure.

DETAILED DESCRIPTION

FIG. 1 shows an example of system 100 generating identifiers forrecommended media assets for multiple users detected to be in a vicinityof user equipment, in accordance with some embodiments of thisdisclosure. System 100 may include user equipment 106 (e.g., atelevision, mobile device, phone, tablet, computer, or any othercomputing device) and sensor 108 (e.g., a camera) communicativelycoupled to (or included as part of) user equipment 106. User equipment106 may include a graphical user interface (GUI), which may include oneor more GUIs 110, 120, enabling users to interact with a contentrecommendation application. User equipment 106, sensor 108, biometricdatabase 116, and user profile database 118 may be communicativelycoupled via a network (e.g., network 308 of FIG. 3 , network 458 of FIG.4 ). As referred to herein, the term “media asset” should be understoodto mean an electronically consumable user asset, such as televisionprogramming, as well as pay-per-view programs, on-demand programs (as invideo-on-demand (VOD) systems), Internet content (e.g., streamingcontent, downloadable content, webcasts, etc.), videos, video clips,audio, playlists, websites, articles, electronic books, blogs, socialmedia, applications, games, and/or any other media or multimedia, and/orcombination of the same. In some embodiments, biometric database 116 anduser profile database 118 may be included in any of server 302, mediacontent source 304, and/or media guidance data source 306 of FIG. 3 .

Users 102 and 104 may be viewing recommended content GUI 110 provided bythe content recommendation application, and sensor 108 may capture inreal time one or more images of users 102 and 104. The contentrecommendation application may analyze image 112 of a face of user 102and image 114 of a face of user 104, in order to identify facialcharacteristics of users 102 and 104. For example, the contentrecommendation application may utilize any suitable facial recognitionalgorithm and/or image processing techniques to identify or extractvarious features (e.g., distance between eyes, distance from chin toforehead, shape of jawline, depth of eye sockets, height of check bones,overall arrangement of facial features, size and shape of facialfeatures, etc.) of the face of user 102 in image 112 and the face ofuser 104 in image 114.

The content recommendation application may compare the identified facialfeatures of user 102 to facial feature information of users stored inbiometric database 116, and may compare the identified facial featuresof user 104 to one or more tables of facial feature informationcorresponding to users stored in biometric database 116. Based on suchcomparisons, the content recommendation application may determinewhether there is a match between identified facial features of users102, 104 and facial features of users stored in the biometric database.In some embodiments, the content recommendation application may computea similarity score for each comparison, and may determine that there isa match if a computed similarity score exceeds a certain threshold.

In some embodiments, the content recommendation application may generatean image signature or facial signature of user 102 and user 104. Forexample, the facial signature may comprise a feature vector includingnumerical values representing the various detected facial features(e.g., a numerical value representing a distance between eyes, anumerical value representing a shape of jawline, etc.) and such featurevector may be compared to feature vectors associated with known faces ofusers in biometric database 116.

The content recommendation application may determine based on the abovecomparison that the identified facial features of image 112 matchbiometric data for user 102, and that the identified facial features ofimage 114 match biometric data for user 104. In response to suchdeterminations, the content recommendation application may retrieve userprofiles for each of user 102 and 104 from user profile database 118.The user profiles may indicate, e.g., various interests of the user,viewing history of the user, prior search queries of the user, priorinteractions with media assets by the user, social media interactions bythe user related to media assets, etc. Although user profile database118 and biometric database 116 are depicted as separate databases, itshould be appreciated that user profile database 118 and biometricdatabase 116 may be a single database.

GUI 120 may be generated for presentation to users 102 and 104,including identifiers for media assets 134, 136, and 138, recommendedbased on the retrieved user profiles of users 102, 104. GUI 120 mayinclude identifier 128 indicating a category of one or more media assetsrecommended for user 102 (“John”), identifier 130 indicating a categoryof one or more media assets recommended for user 104 (“Mike”), andidentifier 132 indicating a category of one or more media assetsrecommended for both user 102 and user 104 (e.g., a blendedrecommendation tailored to appeal to each of user 102 and user 104 bytaking into consideration viewing history and/or interests of each ofuser 102 and user 104). Although FIG. 1 shows a single media assetidentifier for each category being generated for presentation by thecontent recommendation application to avoid overcomplicating thedrawing, it should be appreciated that any number of identifiers formedia assets may be generated for presentation for each category. Theidentifiers for the recommended media assets, and the media assets, maybe retrieved from, e.g., server 302, media content source 304, and/ormedia guidance data source 306 of FIG. 3 .

GUI 120 may provide identifiers 134, 136 of media assets recommended tousers 102, 104, respectively, enabling each user to simultaneously beprovided with a recommended media asset. For example, even if users 102and 104 are accessing the content recommendation application under aprofile associated with only user 102, recommendations tailored to user104 may additionally be provided without requiring any effort form user104 (e.g., since user 104 may be identified based on facial recognition,which may be used to log in to a profile associated with user 104). Insome embodiments, if a user is already accessing his or her profile whena new user is detected by sensor 108, the content recommendationapplication may update GUI 120 to additionally include an identifier forrecommended media assets for the new user. Alternatively, none of theusers may be accessing his or her profile prior to the contentrecommendation application initiating the process shown in system 100.

In some embodiments, options 122, 124, 126 may be selectable by a userto alter presentation of GUI 120. For example, if the contentrecommendation application receives user selection of option 124(associated with user 104, “Mike”), GUI 120 may be updated such that theidentifier for recommended media asset 136 may be moved to a moreprominent position (e.g., switched with the identifier for recommendedmedia asset 134, presented as larger relative to the other identifiers,etc.). Similarly, option 126 may be selected to cause GUI 120 to moreprominently present the identifier for recommended media asset 138relative to the other identifiers.

Media assets in category identifier 132 may be recommended by thecontent recommendation application based on a comparison of media assets134 and 136, and/or based on information in the retrieved user profilesof users 102 and 104 identified via facial recognition and imageprocessing techniques. For example, the content recommendationapplication may recommend media asset 138 at least in part due to mediaasset 138 sharing features with media asset 134 (e.g., each starring theactor Christian Bale) recommended to user 102 and media asset 136 (e.g.,each directed by Christopher Nolan) recommended to user 104. The contentrecommendation application may determine that media asset 138 is a“compromise” recommendation, e.g., while user 104 (“Mike”) may not beinterested in horror movies like media asset 134 (“American Psycho”)recommended to user 102, user 104 still enjoys thrillers (e.g., such asmedia asset 138, “The Dark Knight”), and while user 102 (“John”) prefershorror movies, he also enjoys the actor Christian Bale (cast in bothmedia asset 134 and media asset 138). The content recommendationapplication may generate for presentation a media asset (e.g., fromamong media assets 134, 136, 138) selected by users 102, 104.

In some embodiments, at least one media asset may be recommended undercategory identifier 132 that may not otherwise be recommended to user102 or user 104 under categories 128 and 130, respectively. Additionallyor alternatively, a media asset recommended to one of user 102 and 104may be determined to be suitable as a group recommendation, and/or amedia asset recommended to each of user 102 and 104 may be generated forpresentation as a group recommendation in category 132. Upon receivingselection of content included in category 132, the contentrecommendation application may update the user profiles of at least oneof users 102 and 104 based on the selection. Alternatively, the contentrecommendation application may refrain from updating the profiles of theusers when content is selected from category 132.

In some embodiments, if the content recommendation application does notdetect a user (e.g., user 104) for a predefined period of time (e.g., 5minutes), the content recommendation application may cease generatingfor presentation an identifier associated with media asset 136 for suchuser, and optionally remove category 130 from GUI 120. The contentrecommendation application may additionally or alternatively removecategory 132 in response to failing to detect user 104 after apredefined period of time.

Although sensor 108 is depicted in the example of FIG. 1 as a camera, insome aspects of this disclosure, additional or alternative types ofsensors may be employed in connection with the content recommendationapplication. For example, the content recommendation application mayidentify a voice of a known user by comparing sampled audio (e.g.,detected via a microphone) to an audio signature stored for the user ina database, in order to retrieve recommended content for such user. Insome embodiments, any combination of biometric devices may be used(e.g., to detect a fingerprint of a user, gaze of a user, etc.) in orderto identify a user.

In some embodiments, GUI 120 may be configured to provide a tab option,which enables a user to switch between recommended content for user 102,recommended content for user 104, and recommended content for the group.For example, a first screen may show only content recommended for user102, and a user may select an option (e.g., option 124) to navigate fromthe first screen to a second screen, which may show only contentrecommended for user 104, or an option (e.g., option 126) to navigate toa third screen, which may only show content recommended collectively forthe group.

FIG. 2A shows an example of system 200 performing, based on a detectedemotional indicator, an action related to an identifier for arecommended media asset, in accordance with some embodiments of thisdisclosure. User 202 may be viewing GUI 210 generated for presentationon user equipment 206 to provide user 202 with initial recommended mediaasset 207. In some embodiments, recommended media asset 207 may beprovided based on a user profile of user 202 (e.g., retrieved using thetechniques discussed in FIG. 1 ), and/or based on a detected emotionalindicator of a user. The user may have provided other input (e.g.,entered log-in via a button, a remote control, text or voice) to accesshis or her content recommendation application profile.

While user 202 is viewing GUI 210, sensor 208 (e.g., a camera) maycapture in real time, and analyze, one or more images 212 of user 202,and/or capture in real time and analyze other biometric feedbackreceived from the user (e.g., analyze audio of the user detected by amicrophone, or any other biometric response or combination thereof). Thecontent recommendation application may analyze the one or more images212 to identify or extract information regarding various features in theface of user 202 (e.g., facial expressions, gaze patterns, bodylanguage, position of eyes, mouth, nose, etc.). The identified orextracted features may be compared to one or more tables of facialfeatures and corresponding emotional indicators stored in emotionalindicators database 214 to determine which emotional indicator theidentified or extracted features in the one or more images 212 of user202 correspond to. In some embodiments, the content recommendationapplication may determine a match if comparison results indicate atleast a partial match above a certain threshold (e.g., 50%). In someembodiments, a feature vector may be computed for the identified orextracted features in the one or more images 212 of user 202, andcompared to feature vectors of facial characteristics corresponding torespective emotional states (e.g., happy, interested, neutral, sad,disinterested, surprised) stored in emotional indicator database 214.

In some embodiments, the content recommendation application may computea confidence level (e.g., 80% chance the user is laughing or smiling,75% chance the user is angry) based on the detected facial features orcharacteristics, which may be used in the detecting of an appropriateemotional indicator (e.g., interested, neutral, not interested) withrespect to presented identifiers of recommended media asset 207. In someembodiments, movement patterns by the user may be captured (e.g.,including facial expressions, body language, hand gestures) indetermining an emotional state of the user. For example, analysis ofcaptured image 212 of the user may indicate the user is shaking his orher head no, indicating he or she is not interested in the initial mediaasset recommendation 207.

If the content recommendation application determines there is match 218between the facial features identified or extracted from image 212 ofthe user and features associated with an emotional indicator 216 (e.g.,an emotion of “sad”), the content recommendation application maydetermine that user 202 is unhappy with and otherwise disinterested ininitial recommended asset 207. Thus, the content recommendationapplication may perform an action related to the identifier of mediaasset 207 in accordance with the detected emotional indicator of user202. For example, the content recommendation application may referenceuser profile database 220 in order to obtain media preferences of user202, and may use such media preferences to recommend one or more newmedia assets (e.g., from server 302, media content source 304, and/ormedia guidance data source 306 of FIG. 3 ) that may be of more interestto the user than media asset 207. In some embodiments, the identifierfor media asset 207 that user 202 is determined to be disinterested inmay be replaced in GUI 222 by the identifier for updated media assetrecommendation 236. Alternatively, the user may be prompted as towhether he or she would like to remove media asset 207 from GUI 222and/or be eliminated from impacting future recommendations, or theidentifier for updated media asset recommendation 236 may be added toGUI 222 without removing the identifier for initial media assetrecommendation 207. In some embodiments, emotional indicator database214 and user profile database 220 may be included in any of server 302,media content source 304, media guidance data source 306 of FIG. 3 .

If multiple images of user 202 are captured during a user session, thecontent recommendation application may compare each set of facialcharacteristics associated with respective captured images to determinerespective emotional indicators for each image. Such respectiveemotional indicators may be used to determine an aggregate emotionalindicator of the user during the user session, such as by utilizing oneor more of a variety of techniques (e.g., an average emotional indicatorof the detected emotional indicators over the time period, the mostcommon emotional indicator detected over the time period, the mostrecent emotional indicator detected during the time period, theemotional indicator having the highest confidence score over the timeperiod, or any combination thereof).

In some embodiments, the content recommendation application may wait apredetermined period of time (e.g., 10 seconds) prior to updating amedia asset recommendation based on a detected emotional state of theuser. For example, the content recommendation application may updaterecommended content upon determining that the emotional indicators oversuch predetermined period of time indicate the user is consistently notinterested in recommended content while scrolling throughrecommendations.

In some embodiments, a plurality of identifiers for respective initialrecommended media assets may be presented to the user, and the emotionalindicator of the user may be detected while the user is scrollingthrough the plurality of initial recommended media assets. Prior toperforming an action related to the identifiers, the contentrecommendation application may wait until a predefined time has elapsed(e.g., 10 seconds). If GUI 210 or 222 includes a plurality ofidentifiers for media assets, the content recommendation application maydetermine a media asset of interest based on which identifier ishighlighted by the user via a cursor or selector. If the contentrecommendation application determines user 202 is not interested in thehighlighted media asset (e.g., if the user had highlighted the mediaasset to see more details or a description of the media asset listing),such media asset identifier may be replaced with an identifier for anupdated media asset.

FIG. 2B shows an example of system 201 performing, based on a detectedemotional indicator, an action related to an identifier for arecommended media asset, in accordance with some embodiments of thisdisclosure. The example of FIG. 2B is similar to the example of FIG. 2A,except the content recommendation application, after analyzing one ormore images 224 of user 202, may determine that the identified orextracted facial characteristics of user 202 match 228 storedcharacteristics of a “happy” or interested emotional indicator 226stored in emotional indicator database 214. In this instance, thecontent recommendation application determines the user is interested inone or more media assets currently presented, and may perform a suitableaction (e.g., the content recommendation application may automaticallygenerate for presentation recommended media asset 207 on GUI 232, promptuser 202 to indicate whether he or she desires to consume media asset207, provide a countdown until media asset 207 is to be generated forpresentation, and/or generate for presentation a preview of recommendedmedia asset 207).

In some embodiments, prior to taking action, the content recommendationapplication may wait until the user has exhibited emotional indicator226 for the majority of a time period (e.g., 3 seconds out of 5 seconds)of viewing GUI 210, or the average emotional indicator for the user overa certain time period indicates he or she is interested in the mediaasset. In some embodiments, the content recommendation application mayautomatically add, or prompt the user to add, media asset 207 to his orher watch list or favorite list associated with a user profile of user202, when emotional indicator 226 indicates the user is interested in amedia asset. Additionally or alternatively, the content recommendationapplication may generate for presentation identifiers, at a current timeor a later time, recommending other media assets sharing characteristicswith media asset 207, and/or update the profile of the user based onmedia asset 207.

FIG. 2C shows an example of system 203 performing, based on a detectedemotional indicator, an action related to an identifier for arecommended media asset, in accordance with some embodiments of thisdisclosure. The example of FIG. 2C is similar to the examples of FIGS.2A and 2B, except the content recommendation application, afteranalyzing one or more images 234 of user 202, may determine that theidentified or extracted facial characteristics of user 202 match 235stored characteristics of a “neutral” emotional indicator 237 stored inemotional indicator database 214. In this instance, it may beinconclusive whether user 202 is interested or not in media asset 207,and the content recommendation application may perform one or more ofvarious actions, e.g., refrain from performing an action until a moreconclusive emotional indicator is detected from the user, add anidentifier for another recommended media asset 236 to GUI, show apreview or more information related to media asset 207, etc.

FIG. 2D shows an example of system 203 performing, based on a detectedemotional indicator, an action related to an identifier for arecommended media asset, in accordance with some embodiments of thisdisclosure. In this example, GUI 210 generated for presentation by thecontent recommendation application may be viewed by multiple users 202and 204. The content recommendation application may use techniquessimilar to those discussed in FIGS. 2A-2C to detect respective emotionalindicators associated with each of user 202 and user 204 while GUI 210,including initial recommended media asset 207, is being generated fordisplay. As shown in FIG. 2D, the content recommendation application maydetermine that image 240 of facial characteristics of user 202 matches246 a “sad” or negative emotional indicator 244, and that image 242 offacial characteristics of user 204 matches 250 a “happy”, interested orpositive emotional indicator 248 stored in emotional indicator database214.

Since the emotional indicators of user 202 and 204 conflict (e.g., user204 is interested whereas user 202 is not interested), the contentrecommendation application may perform an action to address theconflict. For example, the content recommendation application maygenerate for display an identifier for one or more updated media assetrecommendations 236 (e.g., based on a user profile of user 202, user204, or a combination thereof, and/or the emotional indicator itself),and subsequently monitor emotional indicators related to the newrecommended media asset. As another example, the content recommendationapplication may generate for presentation a preview of the media asset207, and monitor emotional indicators of users 202 and 204 to determinesubsequent action to be taken. In some embodiments, if one or more ofthe users detected by the content recommendation application does nothave a user profile associated with the media service, other techniquesmay be used to generate for presentation updated recommendations (e.g.,based on trending or popular programming, prompting such user to createa profile and enter his or her interests, etc.). Identifiers forrecommended media assets, and media assets, may be retrieved from, e.g.,server 302, media content source 304, and/or media guidance data source306 of FIG. 3 .

In some embodiments, a selector cursor or highlight icon may be used bythe content recommendation application to determine which recommendedmedia asset the user is reacting to. For example, in GUI 252, if aselector cursor or highlight icon (e.g., being controlled by the uservia input, or placed on a particular media asset when the user beginsaccessing the GUI of media asset identifiers) is associated with theidentifier for recommended media asset 236, the content recommendationapplication may determine that any detected emotional indicators of user202 and 204 correspond to media asset 236. If each of users 202 and 204have the same or similar reactions to a media asset (e.g., there is noconflict in the emotional indicators of the users), an action consistentwith the same or similar emotional indicator may be taken by the contentrecommendation application.

Although the example of FIG. 2D shows two users 202 and 204 interactingwith the content recommendation application, it should be appreciatedthat emotional indicators of any number of users may be detected bysensor 208 and used in performing an action related to an initialrecommended media asset. In some embodiments, an aggregate emotionalindicator of multiple users during the user session may be detected. Forexample, the emotional indicator exhibited by a majority of users in thecaptured images may determine the action to be performed, or an averageemotional indicator for the users in the captured images may determinethe action to be performed. Additionally or alternatively, a priorityuser may be designated (e.g., as the user holding a remote control foruser equipment 206, which may be captured in the one or more images, orthe primary user associated with the particular account for the mediaprovider), such that the emotional indicator of the priority user takesprecedence in determining the aggregate emotional indicator. In someembodiments, an emotional indicator of a newly detected user may bedetected (e.g., for at least a predetermined period of time) and mayimpact the action to be performed. In addition, if the contentrecommendation application detects that a user has exited the vicinityof user equipment (e.g., for at least a predetermined period of time),the emotional indicator associated with such user may be disregarded indetermining an action to be performed related to recommending mediaassets.

In some embodiments, emotional indicator database 214 may storehistorical pictures of users (e.g., tagged or associated with aparticular emotional indicator). When determining emotional indicatorsfor a particular user, the content recommendation application mayperform facial recognition to identify the user, and may compare theimage of the identified user to past images of such user stored inemotional indicator database 214. If the content recommendationapplication determines there is a close match between the image of theuser and an image in emotional indicator database 214 (e.g., asimilarity above a predefined threshold), the content recommendationapplication may determine that the current emotional state of the usercorresponds to the emotional indicator associated with the image storedin emotional indicator database 214.

FIG. 3 shows an illustrative block diagram of a system 300 fordisplaying content, in accordance with some embodiments of thedisclosure. In various aspects, system 300 includes one or more ofserver 302, media content source 304, media guidance data source 306,communication network 308, and one or more computing devices or userequipment 310, e.g., user television equipment 310 a (e.g., a set-topbox), user computer equipment 310 b (e.g., a desktop or laptop), and/orwireless user communications device 310 c (e.g., a smartphone device ortablet). The computing device or user equipment 310 may correspond touser equipment 106 and 206 in FIGS. 1 and 2A-2D, and may include one ormore sensors or devices (e.g., a camera, a microphone, eye scanner,fingerprint scanner, remote control, etc.) to collect biometric data ofusers. Although FIG. 3 shows one of each component, in various examples,system 300 may include fewer than the illustrated components, multiplesof one or more illustrated components, and/or additional components.Communication network 308 may be any type of communication network,e.g., the Internet, a mobile phone network, mobile voice or data network(e.g., a 4G or LTE network), cable network, public switched telephonenetwork, or any combination of two or more of such communicationnetworks. Communication network 308 includes one or more communicationpaths, such as a satellite path, a fiber-optic path, a cable path, apath that supports Internet communications (e.g., IPTV), free-spaceconnections (e.g., for broadcast or other wireless signals), or anyother suitable wired or wireless communication path or combination ofsuch paths. Communication network 308 communicatively couples variouscomponents of system 300 to one another. For instance, server 302 may becommunicatively coupled to media content source 304, media guidance datasource 306, and/or computing device 310 via communication network 308.

In some examples, media content source 304 and media guidance datasource 306 may be integrated as one device. Media content source 304 mayinclude one or more types of content distribution equipment including atelevision distribution facility, cable system headend, satellitedistribution facility, programming sources (e.g., televisionbroadcasters, such as NBC, ABC, HBO, etc.), intermediate distributionfacilities and/or servers, Internet providers, on-demand media servers,and other content providers. NBC is a trademark owned by the NationalBroadcasting Company, Inc.; ABC is a trademark owned by the AmericanBroadcasting Company, Inc.; and HBO is a trademark owned by the Home BoxOffice, Inc. Media content source 304 may be the originator of content(e.g., a television broadcaster, a Web cast provider, etc.) or may notbe the originator of content (e.g., an on-demand content provider, anInternet provider of content of broadcast programs for downloading,etc.). Media content source 304 may include cable sources, satelliteproviders, on-demand providers, Internet providers, over-the-top contentproviders, or other providers of content. Media content source 304 mayalso include a remote media server used to store different types ofcontent (e.g., including video content selected by a user) in a locationremote from computing device 310. Systems and methods for remote storageof content and providing remotely stored content to user equipment arediscussed in greater detail in connection with Ellis et al., U.S. Pat.No. 7,761,892, issued Jul. 20, 2010, which is hereby incorporated byreference herein in its entirety.

Media content source 304 and media guidance data source 306 may providecontent and/or media guidance data to computing device 310 and/or server302 using any suitable approach. In some embodiments, media guidancedata source 306 may provide a stand-alone interactive television programguide that receives program guide data via a data feed (e.g., acontinuous feed or trickle feed). In some examples, media guidance datasource 306 may provide program schedule data and other guidance data tocomputing device 310 on a television channel sideband, using an in-banddigital signal, an out-of-band digital signal, or any other suitabledata transmission technique.

As described in further detail below, server 302 may manage thecommunication of a live content stream (e.g., a live sporting eventbroadcast, a live news broadcast, or the like) and recorded streams frommedia content source 304 to computing device 310 via communicationnetwork 308. For instance, in some embodiments, content from mediacontent source 304 and/or guidance data from media guidance data source306 may be provided to computing device 310 using a client/serverapproach. In such examples, computing device 310 may pull content and/ormedia guidance data from server 302 and/or server 302 may push contentand/or media guidance data to computing device 310. In some embodiments,a client application residing on computing device 310 may initiatesessions with server 302, media content source 304, and/or mediaguidance data source 306 to obtain content and/or guidance data whenneeded, e.g., when the guidance data is out of date or when computingdevice 310 receives a request from the user to receive content orguidance data. In various aspects, server 302 may also be configured todetect events within the live content stream and, based on the detectedevents, control the display of content and/or navigation menu optionsvia computing device 310. Additionally, although FIG. 3 shows mediacontent source 304 and media guidance data source 306 as separate fromserver 302, in some embodiments, media content source 304 and/or mediaguidance data source 306 may be integrated as one device with server302.

Content and/or media guidance data delivered to computing device 310 maybe over-the-top (OTT) content. OTT content delivery allowsInternet-enabled user devices, such as computing device 310, to receivecontent that is transferred over the Internet, including any contentdescribed above, in addition to content received over cable or satelliteconnections. OTT content is delivered via an Internet connectionprovided by an Internet service provider (ISP), but a third partydistributes the content. The ISP may not be responsible for the viewingabilities, copyrights, or redistribution of the content, and maytransfer only IP packets provided by the OTT content provider. Examplesof OTT content providers include FACEBOOK, AMAZON, YOUTUBE, NETFLIX, andHULU, which provide audio and video via IP packets. YouTube is atrademark owned by Google LLC; Netflix is a trademark owned by Netflix,Inc.; Hulu is a trademark owned by Hulu, LLC; Facebook is a trademarkowned by Facebook, Inc.; and Amazon is a trademark owned by Amazon.com,Inc. OTT content providers may also include any other OTT contentprovider. OTT content providers may additionally or alternativelyprovide media guidance data described above. In addition to contentand/or media guidance data, providers of OTT content can distributeapplications (e.g., web-based applications or cloud-based applications),or the content can be displayed by applications stored on computingdevice 310.

FIG. 4 is an illustrative block diagram showing additional details ofthe system 400 (which may be the same as system 300 of FIG. 3 ), inaccordance with some embodiments of the disclosure. In particular,server 401 (e.g., the same server as server 302) includes controlcircuitry 402 and I/O path 408, and control circuitry 402 includesstorage 404 and processing circuitry 406. Computing device 460 (e.g.,one or more of devices 310 a, 310, and 310 c) includes control circuitry410, I/O path 416, speaker 418, display 420 (as well circuitry forgenerating images for display on display 420), and user input interface422. Control circuitry 410 includes storage 412 and processing circuitry414. Control circuitry 402 and/or 410 may be based on any suitableprocessing circuitry such as processing circuitry 406 and/or 414. Asreferred to herein, processing circuitry should be understood to meancircuitry based on one or more microprocessors, microcontrollers,digital signal processors, programmable logic devices,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), etc., and may include a multi-core processor (e.g.,dual-core, quad-core, hexa-core, or any suitable number of cores). Insome embodiments, processing circuitry may be distributed acrossmultiple separate processors, for example, multiple of the same type ofprocessors (e.g., two Intel Core i9 processors) or multiple differentprocessors (e.g., an Intel Core i7 processor and an Intel Core i9processor).

Each of storage 404, storage 412, and/or storages of other components ofsystem 300 (e.g., storages of media content source 454, media guidancedata source 456, and/or the like) may be an electronic storage device.In some embodiments, media content source 454 may be the same as mediacontent source 304. In some embodiments, media guidance data source 456may be the same as media content source 306. As referred to herein, thephrase “electronic storage device” or “storage device” should beunderstood to mean any device for storing electronic data, computersoftware, or firmware, such as random-access memory, read-only memory,hard drives, optical drives, digital video disc (DVD) recorders, compactdisc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D discrecorders, digital video recorders (DVRs, sometimes called a personalvideo recorders, or PVRs), solid state devices, quantum storage devices,gaming consoles, gaming media, or any other suitable fixed or removablestorage devices, and/or any combination of the same. Each of storage404, storage 412, and/or storages of other components of system 400 maybe used to store various types of content, media guidance data, and orother types of data. Non-volatile memory may also be used (e.g., tolaunch a boot-up routine and other instructions). Cloud-based storagemay be used to supplement storages 404, 412 or instead of storages 404,412. In some embodiments, control circuitry 402 and/or 410 executesinstructions for a content recommendation application stored in memory(e.g., storage 404 and/or 412). Specifically, control circuitry 402and/or 410 may be instructed by the content recommendation applicationto perform the functions discussed herein. In some implementations, anyaction performed by control circuitry 402 and/or 410 may be based oninstructions received from the content recommendation application. Forexample, the content recommendation application may be implemented assoftware or a set of executable instructions that may be stored instorage 404 and/or 312 and executed by control circuitry 402 and/or 410.In some embodiments, the content recommendation application may be aclient/server content recommendation application where only a clientcontent recommendation application resides on computing device 460, anda server content recommendation application resides on server 401.

The content recommendation application may be implemented using anysuitable architecture. For example, it may be a stand-alone contentrecommendation application wholly implemented on computing device 460.In such an approach, instructions for the content recommendationapplication are stored locally (e.g., in storage 412), and data for useby the content recommendation application is downloaded on a periodicbasis (e.g., from an out-of-band feed, from an Internet resource, orusing another suitable approach). Control circuitry 410 may retrieveinstructions for the content recommendation application from storage 412and process the instructions to perform the functionality describedherein. Based on the processed instructions, control circuitry 410 maydetermine what action to perform when input is received from user inputinterface 422.

In client/server-based embodiments, control circuitry 410 may includecommunication circuitry suitable for communicating with a contentrecommendation application server (e.g., server 401) or other networksor servers. The instructions for carrying out the functionalitydescribed herein may be stored on the application server. Communicationcircuitry may include a cable modem, an integrated services digitalnetwork (ISDN) modem, a digital subscriber line (DSL) modem, a telephonemodem, an Ethernet card, a wireless modem for communication with otherequipment, or any other suitable communication circuitry. Suchcommunication may involve the Internet or any other suitablecommunication networks or paths (e.g., communication network 458). Insome embodiments, communication network 458 may be the same as network308. In another example of a client/server-based application, controlcircuitry 410 runs a web browser that interprets web pages provided by aremote server (e.g., server 401). For example, the remote server maystore the instructions for the application in a storage device. Theremote server may process the stored instructions using circuitry (e.g.,control circuitry 402) and generate the displays discussed above andbelow. Computing device 460 may receive the displays generated by theremote server and may display the content of the displays locally viadisplay 420. This way, the processing of the instructions is performedremotely (e.g., by server 401) while the resulting displays, such as thedisplay windows described elsewhere herein, are provided locally oncomputing device 460. For example, computing device 460 may includedisplay circuitry (e.g., video card circuitry or combination motherboardand video card circuitry) configured to generate for display the displaywindows. Computing device 460 may receive inputs from the user via inputinterface 422 and transmit those inputs to the remote server forprocessing and generating the corresponding displays.

A user may send instructions to control circuitry 402 and/or 410 usinguser input interface 422. User input interface 422 may be any suitableuser interface, such as a remote control, trackball, keypad, keyboard,touchscreen, touchpad, stylus input, joystick, voice recognitioninterface, or other user input interfaces. User input interface 422 maybe integrated with or combined with display 420, which may be a monitor,television, liquid crystal display (LCD), electronic ink display, or anyother equipment suitable for displaying visual images.

Server 401 and computing device 460 may receive content and data viainput/output (hereinafter “I/O”) path 408 and 416, respectively. Forinstance, I/O path 416 may include circuitry that includes one or moreof communication port configured to receive a live content stream fromserver 401 and/or media content source 454 via a communication network458. Storage 412 may be configured to buffer the received live contentstream for playback, and display 420 may be configured to present thebuffered content, navigation options, alerts, and/or the like via aprimary display window and/or a secondary display window. I/O paths 408,416 may provide content (e.g., a live stream of content, broadcastprogramming, on-demand programming, Internet content, content availableover a local area network (LAN) or wide area network (WAN), and/or othercontent) and data to control circuitry 402, 410. Control circuitry 402,410 may be used to send and receive commands, requests, and othersuitable data using I/O paths 408, 416. I/O paths 408, 416 may connectcontrol circuitry 402, 410 (and specifically processing circuitry 406,414) to one or more communication paths (described below). I/O functionsmay be provided by one or more of these communication paths but areshown as single paths in FIG. 4 to avoid overcomplicating the drawing.

Having described systems 300 and 400, reference is now made to FIG. 5 ,which depicts an illustrative flowchart of process 500 for providingmedia content recommendations that may be implemented by using systems300 and 400, in accordance with some embodiments of the disclosure. Invarious embodiments, the individual steps of process 500 may beimplemented by one or more components of systems 300 and 400. Althoughthe present disclosure may describe certain steps of process 500 (and ofother processes described herein) as being implemented by certaincomponents of systems 300 and 400, this is for purposes of illustrationonly, and it should be understood that other components of systems 300and 400 may implement those steps instead. For example, the steps ofprocess 500 may be executed by server 401 and/or by computing device 460to provide content recommendations.

At 502, control circuitry 410 may identify, by facial recognition,multiple users (e.g., users 102 and 104 of FIG. 1 ) in a vicinity ofuser equipment (e.g., user equipment 106 of FIG. 1 ). Facial recognitionmay be facilitated by capturing one or more images (e.g., images 102 and104 of FIG. 1 ) of each user by way of a sensor (e.g., sensor 108 ofFIG. 1 , which may be an image sensor), and performing image processingand facial recognition techniques on the captured images to extractand/or identify features of the faces of the users. The extracted facialcharacteristics may be compared to facial characteristics of users in adatabase (e.g., biometric database 116 of FIG. 1 ) to determine whetherthere is a match. In some embodiments, upon setting up a user profile,the user may be prompted to provide an image of himself or herself, inorder to populate the database for subsequent matching.

In some embodiments, artificial intelligence (e.g., machine learning)techniques may be employed by control circuitry 410 and/or controlcircuitry 402 in matching facial characteristics of users to known usersin a database (e.g., biometric database 116 of FIG. 1 ). For example, aneural network or convolutional neural network machine learning model(e.g., stored in local memory 412 of FIG. 4 and/or remote server 401 ofFIG. 4 ) may be trained to accept as input two sets of facial featurecharacteristics (e.g., feature vectors), and output a match probability(e.g., by identifying key features or patterns predictive of a match).Training data may include image pairs labeled as matches (e.g., by humanreviewers). The match probability may be compared to a threshold valueto determine whether there is a match between the two sets of facialcharacteristics. Neural networks are discussed in greater detail inconnection with Brehm, U.S. Patent Application Publication No. US2020/0183773 A1, published Jun. 11, 2020, which is hereby incorporatedby reference herein in its entirety. In some embodiments, other machinelearning models may additionally or alternatively be employed (e.g.,classifier algorithms, K-nearest neighbors, etc.)

At 504, control circuitry 410 may query a database (e.g., user profiledatabase 118 of FIG. 1 ) based on the identified users, to obtain userpreferences and/or a viewing history associated with user profiles ofthe identified users (e.g., users 102 and 104 of FIG. 1 ). In someembodiments, the user profile database and the biometric database may bea single database (e.g., at server 302 of FIG. 3 ).

At 506, control circuitry 410 and/or control circuitry 402 may determinerecommended media assets for the identified users (e.g., users 102, 104of FIG. 1 ). In some embodiments, if a particular user does not have auser profile previously registered with a media provider, other factorsmay be taken into account to create a user profile for the user in orderto provide media assets recommendations (e.g., a demographic of the useridentified by the facial recognition, popular or trending programming,etc.).

At 508, the control circuitry may generate for presentation a GUI (e.g.,GUI 120 of FIG. 1 ) including identifiers for each recommended mediaasset (e.g., respective identifiers of media assets 134, 136, 138). Insome embodiments, the recommended media assets may be categorized basedon an associated user (e.g., category 128 may correspond to user 102 ofFIG. 1 , category 130 may correspond to user 104 of FIG. 1 ).

At 510, the control circuitry may receive user selection of one of theidentifiers generated for presentation on a GUI (e.g., GUI 120 of FIG. 1). In some embodiments, the GUI may include selectable options (e.g.,option 128 associated with user 102, option 130 associated with user104) the selection of which enables an identifier for a particular mediaasset recommended to a user (e.g., user 102) to be presented moreprominently than that of another identified user (e.g., user 104). Insome aspects of this disclosure, the selectable option may be a tab totoggle between identifiers of recommended media assets for theidentified users.

If, at 510, user selection of an identifier is received, controlcircuitry generates for presentation the media asset associated with theselected identifier, at 512. If such a selection has not yet beenreceived, the control circuitry may continue to generate forpresentation the identifiers of the media assets, to wait for userselection of one of the identifiers.

In some embodiments, control circuitry 410, communicatively coupled to asensor (e.g., sensor 108 of FIG. 1 ), may continuously check whether oneof the identified users has exited the vicinity of the user equipment,or whether a new user has entered the vicinity of the user equipment.Control circuitry 410 may, if one of the identified users exits thevicinity of the user equipment, cease generating for presentationrecommended content for such user. On the other hand, if controlcircuitry 410 detects that a new user has entered the vicinity of theuser equipment, the process of identifying the user by facialrecognition and generating predicted recommendations for such user basedon a retrieved user profile may be initiated, and the GUI (e.g., GUI 120of FIG. 1 ) may be updated to include a media asset recommendation forthe new user (and optionally a category identifier associated with thenew user).

FIG. 6 depicts an illustrative flowchart of process 600 for providingmedia content recommendations that may be implemented by using systems300 and 400, in accordance with some embodiments of the disclosure. Invarious embodiments, the individual steps of process 600 may beimplemented by one or more components of systems 300 and 400. Althoughthe present disclosure may describe certain steps of process 600 (and ofother processes described herein) as being implemented by certaincomponents of systems 300 and 400, this is for purposes of illustrationonly, and it should be understood that other components of systems 300and 400 may implement those steps instead. For example, the steps ofprocess 600 may be executed by server 401 and/or by computing device 460to provide content recommendations.

Steps 602 and 604 may be performed in a similar fashion to steps 502 and504 described above. Step 606 may be performed in a similar fashion tostep 506 described above, and may additionally include determiningrecommended media assets based on the user profiles of the identifiedusers (e.g., users 102 and 104) blended together. For example, controlcircuitry (e.g., control circuitry 410 and/or control circuitry 402) mayidentify overlap (e.g., a same media asset or a similar media asset,similar interests as between users. etc.) between the user profilesand/or recommended media assets associated with the users (e.g., users102 and 104), and determine a media asset most likely to interest eachof the users.

Step 608 may be performed in a similar fashion to step 508 describedabove, and may additionally include generating for presentation a GUI(e.g., GUI 120 of FIG. 1 ) including one or more identifiers for mediaassets recommended based on the blended user profiles of the users. Theone or more identifiers (e.g., associated with media asset 138 of FIG. 1) may be presented as part of a category (e.g., associated with categoryidentifier 132) associated with the blended user profile recommendation.

At 610, the control circuitry 410 may determine whether one of theidentified users left the vicinity of user equipment (e.g., userequipment 106 of FIG. 1 ). For example, if one of the users is absentfrom the vicinity of the user equipment for a predetermined period oftime (e.g., 2 minutes), the recommended content for such user may beremoved from the GUI, and the blended recommendation may be removed(e.g., if the group includes only 2 users before the user exited) oraltered (e.g., if the group still includes multiple users after the userexits, in order to prevent the interests of the absent user fromimpacting the media assets recommended to the group). In someembodiments, a user may be interested in what his or her friends arewatching (e.g., if a user's friend is known as a movie buff), and mayselect an option to receive recommended media assets (and/or blendedrecommended media assets) based on his or her friend's profile (e.g.,even if the friend is not present or is now absent from the vicinity ofthe user equipment), which may be separately or simultaneously displayedwith his or her own recommendations. If no users left the vicinity ofthe user equipment, processing may continue at step 612.

At 612, the control circuitry may determine that no users have left thevicinity of the user equipment, and may additionally or alternativelydetermine whether a new user entered a vicinity of the user equipment(e.g., in the time period since identifiers of media assets 134, 136,138 of FIG. 1 were generated for presentation). In some embodiments, ifa new user stays in the vicinity of the user equipment for a thresholdperiod of time (e.g., 15 seconds), the control circuitry may performfacial recognition on the new user, and update the GUI to include arecommended media asset for the new user, as well as update the blendedrecommendations for the group of users, taking into account userpreferences of the user profile of the new user in addition to thepreferences of the other identified users in the group. If no users haveleft the vicinity of the user equipment, processing may continue at step614.

At 614, control circuitry 410 may determine whether selection of one ofthe presented identifiers (e.g., an identifier associated with mediaasset 138 of FIG. 1 ) is received. If such a selection is received,processing may move to step 616. If such a selection is not received,the control circuitry may wait until such a selection is detected.

At 616, the control circuitry may generate for presentation the mediaasset (e.g., media asset 138 of FIG. 1 ) associated with selectedidentifier. In some embodiments, selection of the blended recommendedmedia asset causes the user profiles of at least a subset of the groupof users in the vicinity of the user equipment to be updated.

FIG. 7 depicts an illustrative flowchart of process 700 for providingmedia content recommendations that may be implemented by using systems300 and 400, in accordance with some embodiments of the disclosure. Invarious embodiments, the individual steps of process 700 may beimplemented by one or more components of systems 300 and 400. Althoughthe present disclosure may describe certain steps of process 700 (and ofother processes described herein) as being implemented by certaincomponents of systems 300 and 400, this is for purposes of illustrationonly, and it should be understood that other components of systems 300and 400 may implement those steps instead. For example, the steps ofprocess 700 may be executed by server 401 and/or by computing device 460to provide content recommendations.

At 702, control circuitry 410 may generate for presentation to a user(e.g., user 202 of FIGS. 2A-2D) one or more identifiers for respectiveinitial recommended media assets (e.g., media asset 207 of FIG. 2A). Therecommended media asset may be provided based on a user profile of auser retrieved from a database (e.g., user profile database 220 of FIG.2A). In some embodiments, the user and associated user profile may beidentified by a media provider based on biometric data (e.g., facialrecognition) or other input (e.g., remote control, text, voice, etc.).

At 704, control circuitry 410 may be communicatively coupled with asensor (e.g., sensor 208 of FIG. 2A), which captures one or more images(e.g., image 212 of FIG. 2A) of the user (e.g., user 202 of FIG. 2A)while the user is viewing the identifier for the initial recommendedmedia asset (e.g., media asset 207).

At 706, control circuitry 410 may perform image processing and/or facialrecognition techniques on the one or more captured images (e.g., image212 of a face of user 202 of FIG. 2A) to extract or identify variousfeatures (e.g., facial expressions, body language, position of facialfeatures, etc.).

At 708, control circuitry 410 may compare such identified or extractedfacial features to records in a database (e.g., emotional indicatordatabase 214 of FIG. 2A) indicating relationships between facialfeatures and emotional indicators, to determine whether the identifiedor extracted facial features match a particular emotional indicator. Forexample, various numerical values may be assigned to identified orextracted facial characteristics in a feature vector, and such featurevector may be compared to feature vectors stored in database recordsassociated with respective emotional indicators (e.g., interested,neutral, not interested). Control circuitry 410 may compute a similarityscore based on the comparison, and may determine there is a match if thecomputed score is above a certain threshold.

In some aspects of this disclosure (e.g., where multiple images of theuser are captured), control circuitry 410 may employ a variety oftechniques to determine an aggregate emotional indicator of the userduring a particular time period of the user session (e.g., when the userviews a particular identifier of an initial media asset recommendation).The control circuitry may detect respective emotional indicators thatmatch each of detected facial characteristics, and such emotionalindicators may be used to determine an aggregate emotional indicator ofthe user during the particular time period of the user session, based ona variety of techniques (e.g., an average emotional indicator of theuser among the detected emotional indicators over the time period, themost common emotional indicator of the user detected over the timeperiod, the most recent emotional indicator of the user detected duringthe time period, the emotional indicator of the user having the highestconfidence score over the time period, or any combination thereof).

In some embodiments, artificial intelligence (e.g., machine learning)techniques may be employed by control circuitry 402 and/or controlcircuitry 410 in matching facial characteristics of users to emotionalindicators in a database (e.g., emotional indicator database 214 of FIG.2A). For example, a neural network or convolutional neural networkmachine learning model (e.g., stored in local memory 412 of FIG. 4and/or remote server 401 of FIG. 4 ) may be trained to accept as inputtwo sets of facial feature characteristics (e.g., feature vectorsrepresenting the one or more captured images, and feature vectorstypical of a user representing a particular emotion), and output a matchprobability (e.g., by identifying key features or patterns predictive ofa match). Training data may include image pairs labeled as matches(e.g., by human reviewers). The match probability may be compared to athreshold value to determine whether there is a match between the twosets of facial characteristics.

At 710, control circuitry 410 may determine whether the extracted oridentified facial characteristics match a neutral emotional indicator(e.g., emotional indicator 237 of FIG. 2C, which may be stored inemotional indicator database 214). If the control circuitry determinesthere is a match (e.g., match 235 of FIG. 2C) with the neutral emotionalindicator, processing may move to 716. If the control circuitrydetermines there is not a match with the neutral emotional indicator,processing may move to 712. A neutral emotional indicator may correspondto facial characteristics that do not exhibit much emotion (e.g., ablank stare), if the user is looking away from the GUI (e.g., checkinghis or her mobile device), or exhibiting facial characterizes or bodylanguage that do not demonstrate one way or another that the user isinterested or disinterested in the recommended media asset (e.g., mediaasset 207 of FIG. 2C).

At 712, control circuitry 410 may determine whether the extracted oridentified facial characteristics match an interested emotionalindicator (e.g., emotional indicator 226 of FIG. 2B stored in emotionalindicator database 214). If the control circuitry determines there is amatch with the interested emotional indicator, processing may move to716. If the control circuitry determines there is not a match with theinterested emotional indicator, processing may move to 714. Aninterested emotional indicator may correspond to facial characteristicssuggesting the user is happy (e.g., smiling and/or laughing) or excited,and/or movement (e.g., nodding his or her head) suggesting the user isinterested in a particular recommended media asset (e.g., media asset207 in FIG. 2A).

At 714, control circuitry 410 may determine that the user is notinterested in the media asset (e.g., since the comparison in each of 710and 712 may not have resulted in a match). A “not interested” emotionalindicator (e.g., indicator 216 of FIG. 2A) may correspond to facialcharacteristics suggesting the user is unhappy (e.g., frowning or upset)and/or movement (e.g., shaking his or her head), suggesting the user isnot interested in a particular recommended media asset (e.g., mediaasset 207 in FIG. 2A). In some embodiments, control circuitry 410 maydetermine a highest similarity score, computed based on comparisons topossible emotional states (e.g., interested, neutral, not interested),and the detected emotional indicator of the user may correspond to thehighest similarity score.

At 716, control circuitry 410 may perform a suitable action based on thedetected emotional indicator. For example, if control circuitry 410determines at 710 the user is neutral towards the recommended mediaasset (e.g., media asset 207 of FIG. 2C), control circuitry 410 mayperform one or more actions from among a variety of suitable actionsrelated to a currently presented identifier of the recommended mediaasset (e.g., wait for the user to demonstrate a more meaningful emotion,generate for display an identifier for an updated media asset, refrainfrom taking action, etc.). If control circuitry 410 determines at 712the user is interested in the recommended media asset (e.g., media asset207 of FIG. 2B), control circuitry 410 may perform one or more actionsfrom among a variety of suitable actions related to the currentlypresented identifier of the recommended media asset (e.g., cause therecommended media asset to be presented to the user, add the media assetto a user playlist or watch list, remind the user to watch the mediaasset at a later time, present a preview of the media asset, etc.). Ifcontrol circuitry 410 determines at 714 the user is not interested inthe recommended media asset, control circuitry 410 may perform one ormore actions from among a variety of suitable actions related to thecurrently presented identifier of the recommended media asset (e.g.,generate for display an identifier for one or more updated media assets,in addition to or replacing the currently displayed identifier, modifyuser preferences to avoid a media asset with similar characteristicsfrom being recommended to the user again, etc.).

In some embodiments, the detected emotional indicator, along with theretrieved user profile of the user, may be used in selecting an updatedrecommended media asset to present to the user. For example, if controlcircuitry 410 determines the emotional indicator suggests the user issad or angry about the initial recommended media asset, an updatedrecommended media asset may be provided to improve the mood of the user(e.g., a comedy may be recommended to the user, and the user profile ofthe user may also be taken into account in selecting the particularcomedy). In some embodiments, prior to presenting identifiers forrecommended media assets, control circuitry 410 may determine an initialemotional indicator of the user (e.g., when the user accesses a mediaprovider application or turns on the user equipment), and the initialmedia asset recommendation may be based on the initial emotionalindicator of the user (e.g., if the user is in a happy mood, arecommendation for a comedy may be recommended; if control circuitrydetects a user is accompanied by his or her significant other, arecommendation for romantic comedy may be presented; if a mother,father, daughter and son are detected, a recommendation forfamily-friendly content may be presented).

FIG. 8 depicts an illustrative flowchart of process 800 for providingmedia content recommendations that may be implemented by using systems300 and 400, in accordance with some embodiments of the disclosure. Invarious embodiments, the individual steps of process 800 may beimplemented by one or more components of systems 300 and 400. Althoughthe present disclosure may describe certain steps of process 800 (and ofother processes described herein) as being implemented by certaincomponents of systems 300 and 400, this is for purposes of illustrationonly, and it should be understood that other components of systems 300and 400 may implement those steps instead. For example, the steps ofprocess 800 may be executed by server 401 and/or by computing device 460to provide content recommendations.

At 802, control circuitry 410 may identify, by facial recognition,multiple users (e.g., users 202 and 204 of FIG. 2D) in a vicinity ofuser equipment (e.g., user equipment 206 of FIG. 2D). In someembodiments, at least one of such users may be identified by facialrecognition. Facial recognition may be facilitated by capturing one ormore images (e.g., images 240 and 242 of FIG. 2D) of each user by way ofa sensor (e.g., sensor 208 of FIG. 2D, which may be a camera), andperforming image processing and facial recognition techniques on thecaptured images to extract and/or identify features of a face of theusers. The extracted facial characteristics may be compared to facialcharacteristics of users in a database (e.g., biometric database 116 ofFIG. 1 ) to determine whether there is a match. In some embodiments,upon setting up a user profile, the user may be prompted to provide animage of himself or herself, in order to populate the database forsubsequent matching, and control circuitry 410 may use such images toperform facial recognition of users.

Step 804 may be similar to step 702 described above in connection withFIG. 7 , and may additionally include retrieving user profiles for eachof the identified users, and presenting recommended content (e.g.,initial recommended media asset 207) based on aggregate preferences ofthe users (e.g., a media asset having characteristics each of the usersis likely to be interested in).

Step 806 may be similar to step 704 described above in connection withFIG. 7 , and may additionally include capturing images of each of themultiple users (e.g., users 202 and 204 of FIG. 2D) while presenting theidentifier for the initial recommended media asset. Step 808 may besimilar to step 706 described above in connection with FIG. 7 , and mayadditionally include extracting or identifying facial characteristics ofeach of the multiple users from the captured images (e.g., images 240and 242 of FIG. 2D). Step 810 may be similar to step 708 described abovein connection with FIG. 7 , where the comparison to database records mayadditionally be performed for each user and respective images associatedwith each user.

At 812, an aggregate emotional indicator of the multiple users duringthe user session may be detected. For example, the emotional indicatorexhibited by a majority of users in the captured images may determinethe aggregate emotional indicator of the group, or an average emotionalindicator for the users in the captured images may determine theaggregate emotional indicator of the group. Additionally oralternatively, a priority user may be designated (e.g., the user holdinga remote control for user equipment 206, or the primary user associatedwith the particular account for the media provider), such that theemotional indicator of the priority user takes precedence in determiningthe aggregate emotional indicator.

Steps 814-820 of FIG. 8 may be performed in a similar manner to steps710-716, respectively, of FIG. 7 . In some embodiments, preferences ofeach of the detected users may be taken into consideration (e.g., in acase that performing the action related to the identifier for theinitial media asset recommendation corresponds to generating forpresentation an identifier for an updated media asset recommendation).Once a media asset is selected, user profiles of at least a subset ofthe users may be updated based on the selected media asset. In someembodiments, if a user of the plurality of users exits the vicinity ofuser equipment (e.g., user equipment 206), for at least a predeterminedperiod of time (e.g., 10 seconds) the emotional indicator and userprofile associated with that user may be disregarded in determining anaction to be performed (e.g., the aggregate emotional indicator may beupdated for the remaining group members). In some embodiments, if thecontent recommendation application determines a new user has entered thevicinity of the user equipment, the identify of such user may bedetected (e.g., by sensor 208) and used in determining a new aggregateemotional indicator and the action to be performed related to theidentifier of the currently presented media asset (e.g., media asset 207of FIG. 2D).

The processes discussed above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be exemplary and not limiting. Only the claims that follow aremeant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one embodiment may be applied to any other embodimentherein, and flowcharts or examples relating to one embodiment may becombined with any other embodiment in a suitable manner, done indifferent orders, or done in parallel. In addition, the systems andmethods described herein may be performed in real time. It should alsobe noted that the systems and/or methods described above may be appliedto, or used in accordance with, other systems and/or methods.

What is claimed is:
 1. A method comprising: generating for presentation,by control circuitry, to a user an identifier for an initial recommendedmedia asset; capturing, using a sensor, one or more images of the userwhile generating for presentation the identifier for the initialrecommended media asset to the user; identifying, by the controlcircuitry and during the presentation of the identifier of the initialrecommended media asset, an emotional indicator of the user by:comparing the one or more images captured by the sensor to one or moreimages of the user previously stored in a database, determining whetherthere is a similarity above a predefined threshold between the capturedone or more images and the one or more images of the user previouslystored in the database, determining the emotional indicator of the useras corresponding to an emotional indicator associated with the one ormore images of the user previously stored in the database; wherein theinitial recommended media asset is not being played while capturing theone or more images and while identifying the emotional indicator of theuser; and automatically performing, by the control circuitry and basedon the identified emotional indicator having been identified based onthe control circuitry analyzing the one or more images captured by thesensor, an action related to the identifier for the initial recommendedmedia asset.
 2. The method of claim 1, wherein: the identified emotionalindicator indicates the user is not interested in the initialrecommended media asset; and the action comprises generating forpresentation an identifier for an updated recommended media asset. 3.The method of claim 2, further comprising: retrieving a user profileassociated with the user; wherein the updated recommended media assetassociated with the identifier is determined based on the retrieved userprofile and the identified emotional indicator.
 4. The method of claim1, wherein: the identified emotional indicator indicates the user isinterested in the initial recommended media asset; and the actioncomprises generating for presentation the initial recommended mediaasset.
 5. The method of claim 1, wherein identifying the emotionalindicator of the user comprises identifying at least one of a facialexpression of the user or body language of the user.
 6. The method ofclaim 1, further comprising: identifying an initial emotional indicatorof the user prior to generating for presentation the identifier for theinitial recommended media asset; wherein the identifier for the initialrecommended media asset is generated for presentation based on theinitial emotional indicator of the user.
 7. The method of claim 1,further comprising: generating for presentation a plurality ofidentifiers for respective initial recommended media assets, wherein theplurality of identifiers for the respective initial recommended mediaassets includes the identifier for the initial recommended media asset;wherein the emotional indicator of the user is identified whilereceiving a command from the user to scroll through the identifiers ofthe plurality of initial recommended media assets.
 8. The method ofclaim 1, further comprising: storing in memory a table of facialcharacteristics and corresponding emotional indicators; whereinidentifying the emotional indicator of the user by further comprises:identifying facial characteristics of a face of the user in the one ormore captured images; comparing the identified facial characteristics tothe stored facial characteristics; determining, based on the comparison,whether the identified facial characteristics match the stored facialcharacteristics; and in response to determining the identified facialcharacteristics match the stored facial characteristics, determining theemotional indicator of the user is the emotional indicator thatcorresponds to the matched facial characteristic.
 9. The method of claim1, further comprising: identifying a plurality of users in a vicinity ofuser equipment, wherein the user is included in the plurality of users,and at least one of the plurality of users is detected by facialrecognition; capturing one or more images of the plurality of userswhile generating for presentation the identifier for the initialrecommended media asset to the user; identifying, based on the capturedimages, respective emotional indicators of the plurality of users whilegenerating for presentation the identifier for the initial recommendedmedia asset to the user; and determining an aggregate emotionalindicator of the plurality of users, wherein the action to be performedis determined based on the aggregate emotional indicator of theplurality of users.
 10. The method of claim 9, wherein identifyingrespective emotional indicators of the plurality of users comprisesidentifying at least one of facial expressions of the users or bodylanguage of the users.
 11. A system comprising: a sensor configured tocapture one or more images of a user; and control circuitry configuredto: generate for presentation to the user an identifier for an initialrecommended media asset, wherein the one or more images of the user arecaptured, using the sensor, while generating for presentation theidentifier for the initial recommended media asset to the user;identifying, during presentation of the identifier of the initialrecommended media asset, an emotional indicator of the user by:comparing the one or more images captured by the sensor to one or moreimages of the user previously stored in a database, determining whetherthere is a similarity above a predefined threshold between the capturedone or more images and the one or more images of the user previouslystored in the database, determining the emotional indicator of the useras corresponding to an emotional indicator associated with the one ormore images of the user previously stored in the database; wherein theinitial recommended media asset is not being played while capturing theone or more images and while identifying the emotional indicator of theuser; and automatically perform, based on the identified emotionalindicator having been detected based on the control circuitry analyzingthe one or more images captured by the sensor, an action related to theidentifier for the initial recommended media asset.
 12. The system ofclaim 11, wherein: the identified emotional indicator indicates the useris not interested in the initial recommended media asset; and the actioncomprises generating for presentation an identifier for an updatedrecommended media asset.
 13. The system of claim 12, wherein the controlcircuitry is further configured to: retrieve a user profile associatedwith the user; and determine the updated recommended media assetassociated with the identifier based on the retrieved user profile andthe identified emotional indicator.
 14. The system of claim 11, wherein:the identified emotional indicator indicates the user is interested inthe initial recommended media asset; and the action comprises generatingfor presentation the initial recommended media asset.
 15. The system ofclaim 11, wherein the control circuitry is configured to identify theemotional indicator of the user by identifying at least one of a facialexpression of the user or body language of the user.
 16. The system ofclaim 11, wherein the control circuitry is further configured to:identify an initial emotional indicator of the user prior to generatingfor presentation the identifier for the initial recommended media asset;and generate for presentation the identifier for the initial recommendedmedia asset based on the initial emotional indicator of the user. 17.The system of claim 11, wherein the control circuitry is furtherconfigured to: generate for presentation a plurality of identifiers forrespective initial recommended media assets, wherein the plurality ofidentifiers for the respective initial recommended media assets includesthe identifier for the initial recommended media asset; and identify theemotional indicator of the user while receiving a command from the userto scroll through the identifiers of the plurality of initialrecommended media assets.
 18. The system of claim 11, wherein thecontrol circuitry is further configured to: store in memory a table offacial characteristics and corresponding emotional indicators; andidentify the emotional indicator of the user by analyzing the one ormore captured images by: identifying facial characteristics of a face ofthe user in the one or more captured images; comparing the identifiedfacial characteristics to the stored facial characteristics;determining, based on the comparison, whether the identified facialcharacteristics match the stored facial characteristics; and in responseto determining the identified facial characteristics match the storedfacial characteristics, determining the emotional indicator of the useris the emotional indicator that corresponds to the matched facialcharacteristic.
 19. The system of claim 11, wherein the controlcircuitry is further configured to: identify a plurality of users in avicinity of user equipment, wherein the user is included in theplurality of users, and at least one of the plurality of users isdetected by facial recognition; capture one or more images of theplurality of users while generating for presentation the identifier forthe initial recommended media asset to the user; identify, based on thecaptured images, respective emotional indicators of the plurality ofusers while generating for presentation the identifier for the initialrecommended media asset to the user; determine an aggregate emotionalindicator of the plurality of users; and perform the action based on theaggregate emotional indicator of the plurality of users.
 20. The systemof claim 19, wherein identifying respective emotional indicators of theplurality of users comprises identifying at least one of facialexpressions of the users or body language of the users.